Prepare the data in multi-subject trials

Aim

To fix potential identity swaps after tracking multiple subjects with the Deep learning body point detection technique.

This topic applies to

Experiments set as follows: (see Experiment settings)

▪Number of Subjects per Arena: 2.

▪Tracked Features: Center-point, nose-point and tail-base detection.

▪Body Point Detection Technique: Deep learning.

Background information

When EthoVision XT tracks unmarked subjects, there is chance that subject identity is swapped. This could occur for example when two subjects come into contact with each other. In the example below, the dotted line is the true trajectory of the two subjects. After the crossing, the subject orange is labeled as blue, and blue is labeled as orange.

To reduce or entirely remove identity swaps, when using Deep learning, the software works in two steps:

▪Step 1 (Tracking) - The software finds and tracks the subjects’ body points.

▪Step 2 (Data Set Preparation) - The software reviews the tracks and sorts them based on the visual differences between the subjects. This step is carried out after tracking, and automatically when you calculate/visualize the data. As a result, the occurrence of identity swaps is reduced.

Recognition of individuals is based on the differences in the appearance of the subjects’ fur. For how to mark your subjects, see the recommendations in Deep learning: Requirements

Procedure

The Data Set Preparation is done automatically the first time that you calculate the statistics or visualize the tracks. You can also run this procedure manually in the Track Editor.

1.Choose Acquisition > Edit Tracks.

2.Select one trial from the list on the toolbar that is marked with (not yet prepared).

3.Click Prepare Data.

4.Wait until the trial name shows “(ready)”. Depending on a few factors, including the length of the video and its resolution, this process may take some time.

5.You can now edit the tracks or run analysis.

Error messages

If for any reason, the Data Set Preparation procedure fails for at least one trial/arena, the Data Preparation Report shows in which trial and arena that happened, and what the possible causes could be.

▪Not enough data points available to reliably train the network. This occurs if the trial was short, or the two animals were separated for too short time. Another possibility is that the animals were separated, but at least one of them was curled up for long time. In all those cases the software cannot create a high number of reliable images of the two subjects.

In general, we recommend to acquire a trial of at least five minutes, at the maximum sample rate (25 samples/s or higher).

Although the message may sound pessimistic, there are cases when the individual discrimination worked well. For example when the visual difference between the two subject is very clear. In such a situation a relatively small number of video frames may still be ok to create a reliable network for individual discrimination.

Check the identity labels of the two animals and edit the tracks when necessary.

▪No data points available to train the network. This could occur when tracking was so bad because of poor detection (e.g. not enough contrast, or presence of reflections) or because the two subjects were in contact for the entire duration of the trial. In such a situation there are no data points at all available for training the network; individual discrimination fails, no matter how long the trial lasted.

We advise to check the lighting conditions, the contrast between the subjects and the background, and repeat the trial.

▪The subjects look too similar. This occurs when the two subjects are not marked and there is no significant difference in their visual appearance. Check the suggestions about marking animals: Deep learning: Requirements > Individual marking (for two-subject tracking)

Notes

▪The Data Set Preparation may take some time, depending on the length of the video and the power of the GPU.

▪The Data Set Preparation is not applied to trials where the subjects are tracked with the Contour-based technique for body point detection. For the difference between Contour-based and Deep learning, see Body point detection technique.

▪The trial where data preparation failed is still added to your results table. However, there may be significant identity swaps. There are two options:

▪Open the trial in the Track Editor and fix the identity swaps. See Swap subjects

▪Exclude the problematic trials in your Data profile. See Filter tracks

▪Check that you have the latest version of the driver for the secondary graphics card which does most of the work in deep learning - based tracking.